Migration Process and Incident Analysis

1. Objective of migration

The goal was to:

Replace existing RAID storage with larger disks Preserve all media data and permissions Minimize downtime for Jellyfin media services 2. Migration approach Step 1 — New RAID creation Two new disks were configured in PERC as RAID1 The system exposed them as /dev/sdd Step 2 — Filesystem creation A GPT partition table was created ext4 filesystem was applied to the new volume Step 3 — Initial data transfer

Data was copied using:

rsync -aHAX --numeric-ids --progress

From:

/mnt/Media

To:

/mnt/newMedia 3. Secondary migration

A separate dataset was migrated:

/mnt/BonusDisk/JellyfinMedia/Motorvakantie 2024 → /mnt/Media/JellyfinMedia/Motorvakantie 2024

This was done using rsync with preservation flags to maintain:

Ownership Permissions ACLs Extended attributes 4. Issue encountered during migration Symptom

Two MP4 files became unreadable after migration.

Observed error moov atom not found Invalid data found when processing input 5. Investigation outcome Key findings Files existed in both source and destination File sizes were consistent in some cases ffprobe failed on both copies Original playback still worked in media players Root cause

The issue was not filesystem corruption but:

MP4 container metadata inconsistency Likely incomplete or non-finalized file state during copy rsync faithfully copied an already problematic file state 6. Technical explanation

MP4 files rely on a moov atom containing index metadata.

Possible scenarios:

moov located at end of file (camera-style encoding) file copied during incomplete write/finalization metadata not properly finalized before transfer

This causes:

ffprobe failure but partial playback may still succeed in tolerant players 7. Resolution approach

For affected files:

Attempt container rebuild using ffmpeg:

ffmpeg -i input.mp4 -c copy -movflags +faststart output.mp4 If unsuccessful: re-copy from true original source or re-ingest from camera/archive 8. Lessons learned

1. Rsync preserves problems

Rsync does not validate media integrity; it replicates file state exactly.

2. Media must be “finalized” before migration

Files actively written or not fully closed can copy as corrupted artifacts.

3. Verification requires more than size checks Tools like ffprobe or checksum validation are required for media integrity
4. Two-pass rsync is essential for live systems Initial sync during operation Final sync after stopping writes
5. Best-practice migration workflow (refined) Create destination RAID and filesystem Perform initial rsync pass Stop all write processes Perform final rsync pass Validate with: rsync --dry-run media probing (ffprobe) Switch mounts via UUID in /etc/fstab Monitor system before removing old storage